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This talk presented the work done at ORA for NASA-LRC in the design 
and formal verification of a hardware implementation of a scheme for 
attaining interactive consistency (byzantine agreement) among four 
microprocessors. The microprocessors used in the design are an 
updated version of a formally verified 32-bit, instruction-pipelined, 
RISC processor, MiniCayuga . The 4-processor system, which is designed 
under the assumption that the clocks of all the processors are 
synchronized, provides ' 'software control ' ' over the interactive 
consistency operation. Interactive consistency computation is 
supported as an explicit instruction on each of the microprocessors. 

An identical user program executing on each of the processors decides 
when and on what data interactive consistency must be performed. 


This exercise also served as a case study to investigate the 
effectiveness of reusing the technology which had been developed 
during the MiniCayuga effort for verifying synchronous hardware 
designs. MiniCayuga was verified using the verification system Clio 
which was also developed at ORA. To assist in reusing this technology 
a computer-aided specification and verification tool was developed. 
This tool specializes Clio to synchronous hardware designs and 
significantly reduces the tedium involved in verifying such designs . 
The talk presented the tool and described how it was used to specify 
and verify the interactive consistency circuit. 



Summary 


Achievements 

1. Formalization of abstract Byzantine agreement algorithm. 

2. Use of this algorithm to specify a hardware device. 

3. A mechanically checked proof that the device design is correct. 

4. The implementation of the device form the low-level design. 


Limitations 

I . Assumes synchronized behavior of the processes. 
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Objectives of the Work 

• Design an efficient hardware implementa- 
tion for a 4-processor architecture 


• Use verified MiniCayuga’s in the design 


• Verify the design 


• Reuse MiniCayuga verification technology 

— A method of modeling synchronous hard- 
ware designs in the Clio verification sys- 
tem 

— Formalizing a class of properties most 
commonly encountered in verifying de- 
signs 

— A “standard” proof strategy 
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Presentation Outline 

• IC circuit design 

• The computer-aided hardware verification 
tool 

• How we verified it 

• General observations about the effort 
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The Hardware Design: Overview 









Two new instructions: 


ICOP REG - initiates and co-orinates 

IC computation 

MOVE SREG REG - moves special REG to 

general REG 


| | check if voter is free 
Notfree MOVE STATUS REG1 

JIF REG1 Notfree 
ICOP REG2 

I | check if IC computation is complete 
Notre ady MOVE STATUS REG1 

JIF REG1 Not re ady 

I | move the results of IC to general registers 


MOVE SREGO REG 3 
MOVE SREG1 REG4 
MOVE SREG2 REG5 
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The Hardware Design: Overview 


Fault Region 1 Fault 



Fault Region 4 Fault Region 3 








• voter separate from processor: modularity 

• point-to-point connection: electrical iso- 
lation 

• serialize data transfers: number of pins 
Vs. time 

• Fault region: processor, voter, and the 

connections they feed 
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• no absolute indexing scheme for proces- 
sors/voters 

— relative indexing scheme: succ, succ 2 , 

succ^ 

— IC vectors will be stored in the proces- 
sors in the order of their successors 


• Underlying assumption: clocks are syn- 

chronized with at most a bounded skew 

— hold sender's signal stable for one phase 
longer than needed 
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IC System Design Behavior 



• Initiate: draw the attention of voter (1) 

• Load: transfer private values (2) 

• Exchange: exchange received values (6) 

• Compute: compute and store IC vector (3) 
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MiniCayuga Processor: Summary 

• Inspired by Cayuga (Cornell University) 

• 32-bit RISC processor 

• Design characteristics 

— 32 general purpose registers 

— small and simple instruction set 

— 3-stage instruction pipeline: fetch, com- 
pute, writeback 

— delayed jump, pipeline stalling, internal 
forwarding 

— interrupt 
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What do we prove ? 


Assuming 


• every Cayuga-FT is about to execute an 
ICOP, 

• every Voter is ready to vote, and 


• there is at most one faulty region, 


then, 12 cycles later the system state will sat- 
isfy the following conditions: 

• The IC vectors in the processors are iden- 
tical "up to rotation." 


• The IC vectors are correct w.r.t. to the 
processor private values 12 cycles earlier. 



A Computer-Aided Verification Tool 

• Specializes Clio to the domain of finite 
state controller systems 

• Design specification generation 

• Verification condition formulation 

• Automatic proof support 
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Finite State Controller Systems (FSCS) 

• Central Controller + Data Path compo- 
nents 

• Component behavior is specified as a set 
of actions 

• Controller is specified as an FSM which 
schedules a set of actions on the compo- 
nents. 

• Timing Model 

- Every transition corresponds to a clock 
cycle (with multiple phases) 

- An action may have zero or more units 
(phases) of delay 

— Actions are synchronized with state tran- 
sitions 
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Specification technology reused 

• a method of formalizing the intended op- 
erational model of an FSCS in Caliban/Clio 


designspecgen : : 

data-path-structure -> 
controller-structure -> 
controller-schedule -> 

actions-behavior -> design-spec 

Execute : : STATE -> STATE 


“single clock cycle behavior of design” 
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Proof technology shared 

• Form of the most commonly proved con- 
ditions 

— Invariant conditions 




— Advance conditions 

I 




• Proof strategy: "controlled symbolic eval- 
uation (rewriting) with selective case-splits” 
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The Specification Hierarchy 





Rationale for the hierarchy 


• Decompose proofs into manageable units 

• Need for the black level 

— introduce “error” actions 

— type of Execute is different from that 
of action 

• Implication of intermediate levels 

— pro: proof can take “bigger” steps 

— con: must come up with intermediate 
abstract specification 
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Top Level Specification 


I I IcNetState «(INDEX -> FTCstate) , 

I I (INDEX -> Voterstate) , Interrupts>> 

IcNetStep <<ftc,vtr, int:rest» = 

«newf tc ,newvtr ,rest>> 
where newftc index 

= fault _ftc_step index ftc (ftcinput index) 
newvtr index 

= f ault_vtr_step index vtr (vtrinput index) 
ftcinput index 

= make_ftc_in (select_int index int) 

(fault _to_proc index ftc vtr) 

vtrinput index 

= Voterinput index ftc vtr 

(ftcinput index ) 


fault _f tc_step index s in = 

FtCayugaStep (s index) in , “(faulty index) 
byzCayugaStep (s index) in 

fault _vtr_step index s = 

voterstep (s index) , “(faulty index) 
byzstep (s index) 
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Formal Statement of Correctness 


MainTheorem : = 

Preconditions 's' => ResultConsistent ‘s' 

ResultConsistent 's' := 

Consistent 'icvec s (Iterate #12 IcNetStep s) ‘ 

Consistent ' array ' := 

‘ faulty index' = ' False* => 

IndexConsistent 'array* ‘index* 


IndexConsistent 'array* 'index* 

( 'faulty (succ index) *= ‘False *=> 

'(array index) . succ '= ‘ array (succ index)') 

& ('faulty (succ2 index) '= ‘False '=> 

'(array index) .succ2' = 'array (succ2 index) ) 

& ('faulty (succ3 index) '= 'False '=> 

'(array index) . succ3'= 'array (succ3 index) ) 


20 



Preconditions ‘s' := 

Proper_icnet ‘s‘ & Sync ‘LDP1‘ ‘s‘ & All_go ‘s‘ 


Sync ‘cs‘ ‘<<ftc,vtr,inlist»‘ : = 
(‘faulty 0NE‘ = ‘False' => 

‘control (vtr 0NE)'=‘cs‘) 

& (‘faulty TWO' = 'False' => 

‘control (vtr TWO)‘=‘cs‘) 

& (‘faulty THREE' = 'False' => 

'control (vtr THREE) '='cs‘ ) 
& (‘faulty FOUR' = ‘False' => 

‘control (vtr F0UR)‘*'cs') 


All_go ' s ' := 

('faulty ONE' =' False' => 

('go_of (vtr s ONE) '= 'False' & ‘go_signal s 0NE'=‘G0 
& ('faulty TWO '= 'False' => 

( ‘ go_of (vtr s TWO) '= 'False' & ‘go.signal s TW0'=‘G0 
& ('faulty THREE '= 'False' => 

( ' go__of (vtr s THREE) ' = 'False' & ‘go_signal s THREE' 
& (‘faulty FOUR' =' False' => 

(‘go_of (vtr s FOUR) ‘“'False' & 'go_signal s F0UR‘=‘ 
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Preconditions ‘s' 

Proper_icnet ‘s' & Sync 'LDP1' 's' & All_go s 


Sync 'cs' '«ftc , vtr , inlist» ' : = 
('faulty ONE' = 'False' => 

'control (vtr 0NE)' = 'cs ) 

& ('faulty TWO' = 'False' => 

‘control (vtr TW0)'='cs‘) 

& ('faulty THREE' = 'False' => 

'control (vtr THREE) '=' cs ' ) 
& ('faulty FOUR' = 'False — > 

‘control (vtr F0UR)‘='cs') 


All_go 's' : = 

('faulty ONE' =' False' => 

( ' go_of (vtr s ONE) '='False' 


& 'go_signal s ONE' -'GO')) 


& ('faulty TWO '= 'False 
(‘go_of (vtr s TWO) 


=> 

=' False' & 'go_signal s TW0'=‘G0‘)) 


& ('faulty 
('go_of 


THREE '=' False ' => 

(vtr s THREE) '= 'False' 

& ' go.signal s THREE' -'GO' )) 


& ('faulty 
(‘go.of 


FOUR' = 'False' => 

(vtr s FOUR) '= 'False' 

& 'go_signal 


s 


FOUR' =' GO 


)) 
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The proof strategy reused 

" controlled symbolic execution of design" 

1. Instantiate the states of components and 
inputs with appropriate symbolic constants. 

2. Add all the conditions on the constants 
implied by the preconditions of the theo- 
rem as hypothesis. 

3. Symbolically evaluate design. 

4. Try case-splitting on all the conditionals 
automatically. 

5. If either of the previous two steps seem to 
take too long, then case-spilt on the con- 
troller states and inputs before symbolic 
evaluation (step 3). 
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New technology needed 


• Modeling faulty behavior 

• Specification 

- determining the right hierarchy 

- writing intermediate "abstract" spec 

- defining abstraction function (ABS) 

• Proof: “design level properness” implies 
“abstract level properness” 
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General Observations 


• An engineering-oriented verification expe- 
rience 

Lilith — ► MiniCayuga — ► IC circuit 

• Methodology: top-down + bottom-up 

• Level of effort: 1 man year 

— building the tool 

— developing designs 

— verification 
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Verification Effort Milestones 

• formulated a top level correctness state- 
ment 

• designed and verified a simple voter circuit 

• specified voter and processor for a contin- 
uous voting scheme 

• designed and verified second voter design 
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• discovered continuous voting scheme was 
“hard to synchronize” 

• respecified voter and processor for a voting- 
on-demand scheme 

• redesign and reverify voter 

• verified overall system 

• verified processor 
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• To integrate theorem proving based verifi- 
cation technology into the design process 
we need: 

— more machine assistance 

- domain specialization 


• The next step ? 

— A useful way of reporting failed proof 
attempts 

- Interaction with motivated and patient 
engineering design teams and projects 
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